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ABSTRACT 

In an earlier report I4rJ a consistent estimation scheme 
was given for Gaussian Markov random field models. In this 
report we consider some statistical properties of the resulting 
estimate. Specifically, we derive an expression for the asym¬ 
ptotic mean square error of the estimate for a general model 
and compare the efficiency of this estimate with the popular 
coding estimate for a simple first order isotropic model. __. 
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1. Introduction 

Markov random field (MRF) models are of much interest in 
image analysis and processing. For instance, they have been 
used in texture analysis [2,3] and image restoration [4-6]. 

They are also of interest in the analysis of field data [7]. 

MRF models characterize the special nature of the statistical 
dependency of intensity levels over a neighborhood in an image. 

If y(s) denotes the intensity level at location s, then y(s) 
is written as a linear combination of intensity levels {y(s+r), 
r€N} (where N is known as the neighbor set of dependence) and 
additive noise. The members of set N are pairs of integers 
(k,£) not including (0,0). For example, the first order MRF 
model results when N= {(0/1)#(1,0),(0,-1),(—1,0)} and the second 
order MRF model results when N= {(-1,0),(1,0),(0,-1),(0,1),(-1,-1), 
(-1,1), (1,-1),(1,1)}. Each MRF model is characterized by a set of 
linear weights and the variance of the additive noise. 

Suppose we are given an array of intensity level variations 
{y(s),sf^}, ft={s:(i,j), lSi,jSM} and we are interested in fitting 
a Gaussian MRF model to this data. We also assume that the spec¬ 
ific structure of the MRF model characterized by the neighbor set 
N is known. The problem of estimating N has been considered else¬ 
where [1,8]. Three estimation methods, namely the coding method 
[7], the maximum likelihood (ML) method, and the method in [9] 
can be used to obtain estimates of parameters characterizing a 
MRF model. It is the intent of this paper to analyze in some 
detail the statistical properties of the estimate in [1,8]. 









Specifically, we shall derive an expression for the asymptotic 
mean square error of the estimate for an arbitrary MRF model. 
We evaluate this expression for a simple first order isotropic 
MRF model and compare the efficiency of the estimate with the 
popular coding estimate. 
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2. Estimates and their statistical properties in MRF models 
2.1 Model Representation 




Assume that the given image data (y(s)} obeys the MRF model 
in (2.1), with the associated neighbor set N: 


y(s)= E 0 y(s+r)+e(s),sffl, (2.1) 

r€N r 

In (2.1) the neighbor set N is symmetric, i.e., if r€N then 
-rCN. Further, the coefficients satisfy the constraint 0 r =6_ r . 
The stationary noise sequence {e(s)} is partly characterized by 


E(e(s)|y(r))=0 Vs*r 


( 2 . 2 ) 


Using (2.1) and (2.2) one can prove that the noise sequence 
{e(s)> is correlated with the correlation structure 

E(e(s) e(r))*y, s*r 


=-9 s _ r v, (s-r)€N (2.3) 

= 0 otherwise 

It can be shown by methods similar to [10,11] that an observation 
y(s) obeying (2.1) with the conditions (2.3) indeed satisfies 
the Markov condition 

p(y(s) | y(r),r€n, r ^s) 

= p(y (s)| y(s+t),t€N) 

We shall characterize the neighbor set for the MRF model using 
the set N g , which includes members from the non-symmetric half 
of N, i.e., if s€N g then -s?N g and N = {sss€N g )u{-s:s€N s }. A 
sufficient condition to ensure stationarity is [7] 

Si:9 k,Jl z l z 2 < 1 whenever |z 1 | ■ | z 2 | « 1 










2.2 A consistent estimation scheme [1/8 ]: 
Consider the estimates 


0* = [E q(s) q T (s)] -1 (E q(s) y(s)) (2.4) 

fi-' - ~ fi j. " 

and 

v* = E (y(s)- 9 * T q(s) ) 2 (2.5) 

M 2 fi j. " 


where 


n _ = n - 


and 


fi = {s = (i,j): s6fi and (s+r)/fi 

Jj 

for at least one r€N} 

We now state a theorem regarding the consistency of the estimate 

9* and give an expression for the asymptotic variance of the 

* 

estimate 0 . An expression for the asymptotic variance of 0* for 
an isotropic MRF model with N g = { (0,1),(1,0)} is also given. 


Theorem 1 : Let y(s), s*fi be the set of observations obeying 
the MRF model (2.1). Then 

(i) The estimate 0* is asymptotically consistent. 

(ii) The asymptotic covariance matrix of 0* is 
E(9-9*) (e-e*) T = -Jv Q -1 + 2 v 2 (Q T Q) _1 

m2 ~ ~ 

( 2 . 6 ) 


£5 


-1 E E 


s r ns-r)!:,, 1 ?' 1 ' 1 ' 
(s-r )€N 


where 






(iii) For the isotropic conditional model with N g = 
{(0,1),(1,0)}, the asymptotic expected mean square error is 


, 2e 2 (l-46a. ft ) 2 

j— 


e( 8 - e*) 


(2.7) 


4M a 


1,0 


l l,0 


= cov(y(s),y(s+(l,0))) 
cov(y (s)) 


( 2 . 8 ) 


The elements of matrices Q and T are functions of normalized 

~ ~r, s 

autocorrelation coefficients The proof is given in Appen¬ 

dix I. 

Although 0* is a consistent estimate of 0, it is not very 
efficient. We compare the efficiency of this estimate with 
efficiencies of the coding estimate for a simple MRF model with 
Ng = {(0,1),(1,0)}. The exact ML estimate is obtained by assuming 
a toroidal lattice representation for y(s) and maximizing the 
resulting log likelihood function by using the Newton-Raphson 
procedure. Equation (2.4) can also be used for toroidal lattice 
representation by summing overH instead of n . The resulting 
difference in the numerical value is negligible for sufficiently 
large M. 









2.3 Comparison of estimates 


We compare the asymptotic variance of the estimate (2.4) 
with the asymptotic variances of the coding estimate and ML 
estimate for the isotropic conditional models with N g = { ( 0 , 1 ),( 
From [9], the asymptotic variance of the coding estimate 6 C is 

6(1-400, J 


M 2v ar ( e c ) - - ra -—^ 


(2.9) 


1,0 


Also from [9], the variance of the ML estimate 8 „_ is 

MX* 


where 


Var( 0 ML ) = 


0.5 


( 2 . 10 ) 


M (I(9) - 4VJ q (0)) 


X( 0 ) = 1 jj (cos x+cos y) 2 dxdy 

4ir 2 0 (1-20 (cos x +cos y) ) 


and 


2 7T 

/ax _ 1 tr cos(sx+ty)dxdy 

v st (0) - ^2 Ji ( 1 - 20 (cos x+ cos y) 


Tabulated values of V 1 Q (0), Q and 1(0) are available in [9] 
for different values of 0. Using these values, and (2.7), (2.9 
(2.10), Columns 2-4 of Table I are computed. The asymptotic 
efficiencies in columns 5 and 6 are defined by 


eff( 0 C ) 


Var< » ML> 
Var(9 c i 


and 

Var < 9 ML> 
Var (6 *) 


1 , 0 )} . 


and 


1 


eff( 0 *) 






It is evident that the estimate 0* computed using (2.4) is 
more efficient than the coding estimate but is not as good 
as the ML estimate. Note that column 5 is available in [9]. 
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Appendix I 


proof of Theorem T_: 
i) We have from (2.4) 


9* = C^q(s)q T (s)]' 1 ( £q(s)y(s)> 


( 1 ) 


Substituting for y(s) from (2.1) and simplifying, we have 


[^q(s)q T (s)](6*-«) = ^q(s)e(s) 


( 2 ) 


Since E(q(s)e(s)) = 0 by (2.2) and C ^q(s)q T (s)D is a positive definite 


matrix, the consistency of the estimate e follows. 


C*3 


ii) To make our calculations easy we assume from now on thate(s)is normally 
distributed. Multiplying (2) by its transpose and taking expectations, 
we have 


EC]T)q($)q T (s) (e*-9)(e*~8 ] r (^3q(s)q l (s)) , 3 


T,_\% T. 


= EC q(s)e(s) 53 (q(r)e(r))^3 


(3) 


= 535Z E *3< s)e * s)) E(q T (r)e(r)> 
s r 

+5353 E(g(s)e(r>> E(g T (r)e(s>) 
s r 










(4) 


♦££ E(e(s>e<r)) E( g (r) 9 T(s)> 


s r 


s i + ii + Ills where 


I = 0 , 


by using (2.2); 


11 


= 2 I , by using (2.2)/ 


(5) 


( 6 ) 


(7) 


mxm 


III = v ]T) E(q(s)q T (s)) 
s 


- » EE \,-r> E( 9 tr>9 T <s» 

r s 

(s-r)cN 











for large values of M 

q(s)q T (s) = Q + n(M), 
M s ~ 


where n(M) is such that 


ECn CM)) 


■"(=■) 


Using (11) we obtain 
(LHS of (3))/M 4 = 


EC(Q ♦ n(M))(e*-e)(e*-e) T (Q + n(M)) T ] 


* Q E<e*-0)(0*-0) T q t + 0(1/M 2 ), 


Substitution of (10) and (13) into (3) yields 


E(0*-0)(0*-0) T = Cv Q -1 + 2v 2 (Q 2 ) _1 
' ' ~ ' f-T 


72 i' EE Ir, s <S>-' * «(1/H N 

n s r 

(s-r)cN 


Q.E.O. 


c) For the isotropic conditional model with = <(0,1)>(1,0)>, (14) 


reduces to 


E(0*-e) 2 = ^- \ -=• C4v* 

FT (E(q^(s))^ 










(15) 


♦ vE(q z (s)) - 8v £53 E(q(s)q(r))3 

s r 

(s-r)cN 


where 


q(s) = y(s+(0,1)) + yCs+<0,-1>> 


+ y(s+(1 / 0)> + y(s+<-1,0>> 


Let 


Y k L = ECy(s)y(s+(k,l))3 


Note 


1 


(16) 


(17) 


Y k,i = T -k ri and Y i,k = Y k,i <18) 

Express the higher order correlations Yg y Y2 g and Yj g in 








Y 0,0 " Y 1,1 


( 22 ) 


y 


2,0 


1 

• Y 1,0 


r 1,-1 


y i,- 2 = ?e Y i,-i ~ Y i,o <23) 

Equations (19-23) can be obtained by multiplying y(s) by appropriately 
shifted y(s+(k,l)) and taking expectations. 

Consider the various terms in (15). 


ECq^(s)) 


(24) 


EE E(q(s)q(r)) = 4C9 t 1#q + y JQ ♦ 3Y 2J + 3Y, t _ 2 l 
(s-r)eN 

* tC ‘ T 1,0 * ^2 Y 1,0 - T T 0 , 0 3 

Substitution of (24) and (25) in (15) yields 


E(e*-e) 2 = 1- —— C4v 2 

" ’<0 


which on using (19) and q = 


* 2 2e 2 <l-4ea 1 n) 2 " 

» -«> z =- liM — 


E(9-9) = 




+ 4VY o,o - 

T 1 , 0 /T 1,1 9 1ves 


(25) 





































References 


1) R. Chellappa, "Fitting Markov random field models to 
images," Technical Report TR-994, Computer Science 
Center, University of Maryland, College Park, Maryland, 
20742, Jan. 1981. 

2) M. Hassner and J. Sklansky, "The use of Markov random 
fields as models of textures," Computer Graphics Image 
Proc., Vol. 12, pp. 357-370, April 1980. 

3) G.R. Cross, "Markov random field texture models," Technical 
Report No. 80-02, Department of Computer Science, Michigan 
State University, East Lansing, Michigan, 1980. 

4) R.L. Kashyap and R. Chellappa, "Image restoration using 
random fields," Proc. of the Eighteenth Annual Allerton 
Conf. on Comm., Control and Computing, University of 
Illinois, Urbana, Ill., pp. 856-965, Oct. 1980. 

5) J.W. Woods, "Markov image modeling," IEEE Trans. Automat. 
Contr., Vol. AC-23, pp. 846-850, Oct. 1978. 

6) A.K. Jain, "A fast Karhunen-Loeve transform for digital 
restoration of images degraded by white and colored noise/' 
IEEE Trans. Computers, Vol. C-26, pp. 560-571, June 1977. 

7) J.E. Besag, "Spatial interaction and statistical analysis 
of lattice systems," J. Royal Stat. Soc., Ser. B, Vol. B-36 
pp. 192-236, 1974. 

8) R.L. Kashyap and R. Chellappa, "Estimation and choice of 
neighbors in spatial interaction models of images" (sub¬ 
mitted for publication). 

9) P.A.P. Moran and J.E. Besag, "On the estimation and testing 
of spatial interaction in Gaussian lattices," Biometrika, 
Vol. 62, pp. 555-562, 1975. 

10) J.W. Woods, "Two-dimensional discrete Markov random fields, 
IEEE Trans. Information Theory, Vol. 18, pp. 232-240, March 
1972. 

11) R.L. Kashyap, "Random field models on finite lattices for 
finite images" (presented at the Symposium on Information 
Sciences, Baltimore, MD, March , 1981). 











SECURITY CLASSIFICATION of THIS FACE (Whit Dio Entered) 


REPORT DOCUMENTATION PAGE 


refort number 


'.Vi M. 


AFOSR-TR- 8 1 -0 5 56 





1 




READ INSTRUCTIONS 
BEFORE COMPLETING FORM 


>• recipient's catalog number 




«. title (ltd Submit) 


S. TYPE OF REPORT S FERIOO COVERED 


ON AN ESTIMATION SCHEME FOR GAUSS MARKOV Technical 

RANDOM FIELD MODELS s. performing oro. report number 

TR-1040 


s. contract or grant NUMBER?*) 


7. author?*) 


R. Chellappa 


AFOSR-77-3271 


s. performing organization name and address 

Computer Vision Laboratory, Computer 
Science Center, University of Maryland 
College Park, MD 20742 


it. controlling office name ano address 

Math. & Info. Sciences, AFOSR/NM 
Bolling AFB 

Washington, DC 20332 _ 


4. monitoring AGENCY NAME t AOORESS (II dllltrltl Iron Controlling Ollleo) 


10. PROGRAM ELEMENT. PROJECT. TASK 
AREA * WORK UNIT NUMBERS 

Q'/ZCrZ/^ 

J2 .? O 


12. REPORT DATE 

April 1981 


IS. NUMBER OF PAGES 

15 


IS. SECURITY CLASS, (ol thlt report) 

Unclassified 


IS*. OECLASSIFICATION/ DOWNGRADING 
SCHEDULE 


DISTRIBUTION STATEMENT ?•« thlt Report) 


Approved for public release; distribution unlimited. 


17. DISTRIBUTION STATEMENT (ol tho mbotroct ontorod In Block 70. II dlllortnt liom Report) 



IS. KEY WOROS (Continue on reroree tide II noceetery end Identity by block number) 


Image processing 
Image models 
Random fields 
Estimation 


20. ABSTRACT (Continue on rente tide II nocoetoey end Identity by block number) 

In an earlier report [1] a consistent estimation scheme was given 
for Gaussian Markov random field models. In this report we con- 
side some statistical properties of the resulting estimate. 
Specifically, we derive an expression for the asymptotic mean 
square error of the estimate for a general model and compare the 
efficiency of this estimate with the popular coding estimate for 
a simple first order isotropic model. 


DD ,X*ts 1473 


EDITION OF I NOV •• IS OBSOLETE 


SECURITY CLASSIFICATION OF This PAGE ?W>i*i» Die Entered) 
























